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Let that two types of recognition objects exist. Let that 
probabilities find these two types are known. Two variables 
that characterizes object exist and conditional probabilities to 
find one from two types of object exist as function every one 
variable. The best approximation for conditional probabilities 
as function of the two variables is necessary to find. The fuzzy 
logic usually used is not relevant. Cases of larger number of 
types and variables are considered. 

PACS numbers 47.27.Gs, 47.27.Jv, 05.40. +j 



I. INTRODUCTION 

A. Statement of the problem. 

Let A be a random variable, with values in set 0, 1. 
Assume that the apriori probability P{A) — P(A = 1) 
is known and denote it by 0. Let Xx, X 2 be two random 
variables, with values in some set, say ] — 00; +oo[. We 
are given the following informational = x\ and X 2 = x 2 
(obtained though measurement). Furthermore, we have 
two system - "classifiers" , which given x\ and x 2 pro- 
duce: 



P{A = 1/X X 



Xx, 



P{A/ Xl 



P{A = 1/X 2 = x 2 ) = P(A/x 2 ) = 13 



(1) 



(2) 



We wish to estimate the probability P(A = 1/Xi = 
xi,X 2 = x 2 ) = P(A/xi, x 2 ) in terms of a, j3 and 6. More 
specifically we wish to find a function T opt (a, (3, 0) which 
on the average is the best approximation for P{A/x\, x 2 ) 
in a sense to be defined explicitly in the sequel (see Fig 
!•)• 



II. NOTATION AND PRELIMINARIES 

Px 1 ,x 2 ( x i, x -) - joint PDF (probability density func- 
tion) of Xx and X 2 . Px u x 2 /a(xi,x 2 ) = h(xi,x 2 ) -joint 
PDF of Xx and X 2 , given A = 1. In terms of h(xx,x 2 ) 
and 9 we may write P{A/x\, x 2 ) as follows: 



P(A/xx,x 2 ) = 



9h(xi,x 2 ) 



8h(xi,x 2 ) + (1 - 0)h(xi,x 2 ) 



(3) 



where h(xi,x 2 ) = p Xi X2 ^(xx,x 2 )- joint PDF of Xx 
and X 2 , given A — 0. 
We have: 



Px 1 {xx) 



Px 2 {x 2 ) 



+ OO 



Px u x 2 {xx,x 2 )dx 2 



Px 1 ,x 2 {xx,x 2 )dx 1 



hx{xx) = Px 1 /a{xi) 



h 2 {x 2 ) = Px 2 /a{x 2 ) 



+ CO 



+ CO 



(4) 



(5) 



h(xx,x 2 )dx 2 (6) 



h{xx,x 2 )dxx (7) 



hx{xx) = P Xi /a( x i) = / h{xx,x 2 )dx 2 (8) 



h 2 {x 2 ) = P X2 /a{ x -) = / h{xx,x 2 )dxx (9) 



B. Importance of the problem 

The problem is important for artificial intellect and 
can be widely used for image recognition. 



III. GENERIC FORM OF P(A/X 1 ,X 2 ) 

Define the function g{xx, x 2 ) and g{{xx, x 2 ) 

h{xx,x 2 ) 



g{xx,x 2 ) = 



hx{xx)h 2 {x 2 ) 



(10) 



1 



g(xi,x 2 ) 



h(x u x 2 ) 



(11) 



i.e. 



h 1 (xi)h 2 (x 2 ) 

Note that if Xi and X 2 are conditionally independent, 

h(xi,x 2 ) = Px 1 x 2 /a(xi,x 2 ) = Px 1 /a{x\)px 2 /a{x 2 ) 
hi{x 1 )h i {x 2 ) (12) 



then 



g(xi,x 2 ) = g(x!,x 2 ) = 1 



(13) 



Define the following monotonously nondecreasing 
probability distribution functions 



ffi(zi) 



h\(z)dz 



/X 2 
h 2 (z)dz 
-OO 



Hi( Xl ) 



H 2 {x 2 ) 



h\{z)dz 



h 2 (z)dz 



(14) 



(15) 



(16) 



(17) 



Note that since Hi(xi), H 2 (x 2 ), Hi(x\) and H 2 (x 2 ) 
are monotonous (At this point one could assume 
that hi{xi),h 2 (x 2 ),h\(x\),h 2 {x 2 ) > 0, so that 
H\(x\), H 2 {x 2 ), H±(xi) and H 2 (x 2 ) are monotonously 
increasing. This restriction will be shown to be super- 
fluous in the sequel.) , there exist the inverse func- 
tions H^ 1 (xi), H^ 1 ^), H 1 (xi)a,ndH 2 (x 2 ). We may 
therefore define: 



J(o,6)=^(F 1 1 (o),H a 1 (6)) (19) 
For the sake of brevity we shall henceforth denote 
J=J(H 1 (x 1 ),H 2 {x 2 )) = 

giH-^H^x^^H-HH^))) = g( Xl ,x 2 ) (20) 



J = J{H 1 (x 1 ),H 2 {x 2 )) = 

g(H;\H 1 (x 1 )),H~\H 2 (x 2 )))=g(x 1 ,x 2 ) (21) 
By the definition 

h(xi,x 2 ) = Jhi(xi)h 2 (x 2 ) (22) 

h(xi,x 2 ) = Jhi(xi)h 2 (x 2 ) (23) 



We now have: 
hx{xi) = p Xi /a(xi) 

h 2 {x 2 ) = Px 2 /a(x 2 ) = 



px 1 {xx)P{A/x\) _ ap Xl {xi) 



P(A) 



(24) 



Px 2 (x 2 )P{A/x 2 ) /3px 2 (x 2 ) 



hi{xi) = P Xi/ a{xi) 
(1 - a)p Xl (xi) 



1 - i 



h 2 {x 2 ) = P X2/ a(x 2 ) 
(1 - a)px 2 (x 2 ) 



P(A) 



p Xl (x 1 )P(A/x 1 ) 
P(A) 



Px 2 (x 2 )P(A/x 2 ) 
P(A) 



(25) 



(26) 



1 - 

Hence, from (|22D,(|2J 



h(xi,x 2 ) = J 



a/3p Xl (xi)px 2 (x 2 ) 



(27) 



(28) 



t, \ - (1 - a)(l - (3)p Xl (xi)px 2 (x 2 ) . . 
h(xi,x 2 ) = J — (29) 



Now from (g 
P{A/x x ,x 2 ) = 



(i- Of 



jaf3p Xl (xi)px 2 (x 2 ) 



ja(3p Xl (x 1 )px 2 (x 2 ) + 7i30)(l - «)(1 - P)px x (xi)px 2 (x 2 ) 



J{a,b)=g{H^\a),H^{b)) (18) = 



a(3 



(30) 



Note that in case of conditional independence J = 
J = 1 and (|o|) becomes the exact solution T(a,{3, 9) = 
P(A/xi,x 2 ). 



IV. RESTRICTIONS ON THE FUNCTIONS 

J(A,B) AND J(A,B) 



We have 



hi(xi) = / J(H 1 (x 1 ),H 2 (x 2 ))hi(xi)h 2 (x 2 )dx 2 . 

J — OO 

(31) 

Hence 



2 



1 = 

-1 



J (Hi (xi ) , H 2 (x 2 ))h 2 (x 2 )dx 2 

J 

J(H 1 {x 1 ),H 2 {x 2 ))dH 2 (x 2 ) 

10 

Thus we have the following condition 



/ J(a,b)db=l 
Jo 



and analogously 



J(a, b)da = 1 



Similarly we obtain: 



Obviously 



J(a, b)da = 1 



J(a, b)db = 1 



J(a,b),J(a,b) > 



(32) 



(33) 



(34) 



(35) 



(36) 



Px 1 x 2 (xi,x 2 ) = 6h(xi,x 2 ) + (1 - 9)h(xi,x 2 ) 
6Jhi(xi)h 2 (x 2 ) + (1 - e)lhi{xi)h 2 (x 2 ) = 
.Ja/3 J{l-a){l-P), 



(1-9) 



]px 1 {xi)px 2 (x 2 ) 



\\T(a,(3,0)-P(A/x 1 ,x 2 )\\ 

px 1 (xi)px 2 (x 2 ) 

Jaj3 J(l-a)(l-/3) 
(T - J) J 
(T(a, /3, (9) - P(A/x 1 ,x 2 )) 2 dx 1 dx 2 
' '" 1 r Ja/3 7(l-a)(l-/3). 



CW ^ (l-») 
(r(a, /3, 0) - P(A/xi, x 2 )fdF 1 (x x )dF 2 {x 2 ) 



where 



Fx(xx) = / p Xl (z)dz 



F 2 {x 2 ) 



px 2 (z)dz 



(40) 



(41) 



(42) 



(43) 



J(a, 6)dad6 = 1 . (37) 



J(a, b)dadb 



The set of all the solutions of (|3J) , (|3j) , (|3J) , (g6|) , (|3 
together with (|30| ) determines the set of all possible real- 
izations of P(A/x\ 1 x 2 ). 



An example of a solution of (|33|) , (|34|) (mrf (|36|),(37). 
Let p(x) be a function such that p(x) > and 
lo p{x)dx = 1 
Then 



J(a, b) = p(a — b) ,a>b 
J(a, b) = p(a — 6+1) ,a<b 



(38) 



satisfies (|33D,(|3j) and (|3j),(| 



V. DEFINITION OF DISTANCE 

We define the distance between the proposed approx- 
imation of P(A/x\, x 2 ),- T(a,/3,9) and the actual func- 
tion P(A/xi,x 2 ) as follows: 

\\T(a,p,9)-P(A/x u x 2 )\\ = 

+ 00 

Px 1 x 2 (xi, x 2 ) 



[T(a, (3, 9) - P{A/x 1 , x 2 )] 2 d Xl dx 2 



(39) 



Now we have from (f22j),(|23|) and (N),(E5j), <M)M 



VI. RESTRAINTS FOR BASIC FUNCTIONS 

We will consider in further all functions with argu- 
ments 1 > F1,F2 > 0, but not xi,x 2 . We have six 
function of F1,F2, that define (|l|): J,~J, Hx,H 2 ,a, ft. 
Let us to write the other function by help these function 
and find restraints for there functions. 

(i) 



a = P(A/ Xl ) = 9h 1 (x 1 )/ PXl (x 1 ) = 



By the same way 



} dH 2 
dFi 



Restraints for function Hi, H 2 are next: 

H 1 {l) = H 2 {l) = l 
H 1 (0)=H 2 (0) = 



„. ri =e ^±,[j = 9^< i 



dFi 



< 9 < 1 



dF 2 



} dHi 
dF 1 



(44) 



(45) 



(46) 
(47) 
(48) 



(ii) 



3 



apxi (2:1) 



1-6 1 
By the same way 



-Hi(xi) 



(iii) 



H 2 {x 2 ) = YZq~ J^H 2 (x 2 ) 



J(H 1 (F 1 ),H 2 (F 2 )) : 
J(H 1 (F 1 ),H 2 (F 2 ))>0 



J(a,b)db = 1 



J(a, b)da = 1 



J(H 1 (F 1 ),H 2 (F 2 )) : 
1{H 1 {F 1 ) : H 2 {F 2 )) >0 

J(a, = 1 



J(a, 6)<ia = 1 



(iv) 



Ja/3 



P(A/xi,a;2) 



M _i_ J(i-qQ(i-/3) 
e "•" 1-0 



(49) 



(50) 



(51) 



(52) 



(53) 



Then from(|53j) and (|4lj) 

\\T(a,0,9)-P(A/x 1 ,x 2 )\\ = 

[ [ C(T{a,f3,6)-D/C) 2 dF l dF 2 = 

Jo Jo 

f f dFidF 2 & + T 2 (a,(3,6)C - 2T(a, /3,6)D] (57) 

Jo Jo k 

Thus 

min r ^^F[\\T(a, @,6) - P{A/x x , x 2 )\\] = 

min r ( a ,i3,e)F[ / / dF x dF 2 
Jo Jo 

+ T 2 (a, (3, 0)C - 2T(a, /3, 9)D]] = 

f 1 f 1 D 2 

minr( a ,f3,e) F [j J dF 1 dF 2 [—}] + 

[ f dF x dF 2 
10 Jo 



[r 2 (a, (3, 6)C - 2T(a, [3, 9)D]} = 

Const + TniTlYia.fjfi) 

[T 2 {a,(3,6)C - 2T(a,f3,9)D]} 

It remains to calculate the expected value in (58) 
We have by obvious assumptions 



r-i pi 
: F\ I / dF ld F 2 
10 Jo 



(58) 



Pj,J,a,P,H 1 ,H2/F u F 2 ( J >J> a 'P> H ll H 2 /F 1 ,F 2 ) = 

Pj/Hi,Hz ( J /Hi, H 2)(>j/H 1 jj 2 ( J /Hi, H 2 ) 

Pa/F 1 ( a / F l)PH 1 /a,F 1 (Hl/a,F 1 ) 

Pp /F2 ((3/F 2 )p H2/0tF2 (H 2 /P,F 2 ) (59) 



VII. OPTIMIZATION 

We shell find the best approximation T(a, (3, 6) as fol- 
lows 

minr( a ,p,8)F[\\T(a, [3,6) - P{A/x\, x 2 )\\] — > Y(a,f3,0) 

(54) 

where the expected value F[...] is taken with re- 
spect to the joint PDF of possible realizations of: 
J, J, a, (3, Hi,H 2 for given F± and F 2 . 

For the sake of brevity we denote: 



G 



Jaf3 J(l-a)(l-(3) 



6 



D = 



Ja/3 



(55) 



(56) 



A. Lemma 1 



F[J{a,b)} 



P.J(a,b)/a,b{ J ( a : b )/ a > t>)J(a, fydJ = 1 

(60) 



+ 00 



F[J(a, b)} = / Pj^ b)/a ^J(a, b)/a, b)J(a, b)dJ = 1 



(61) 



Proof: 



Let us consider function: p,j( a .b)/a.b Function J(a,b) 
is defined on the square < a,b < 1 Let us make sam- 
pling of function J on this square by its dividing on small 
squares (i, j) and define value of the function on ev- 
ery square i,j. Restraints for function J (***) can be 
written 



Ji j > 



(62) 



4 



1 N 

- y Jn = i 

i=i 
1 - 



(63) 



(64) 



here i = 1, ...,N, j = 1, ...,N 

All matrixes (Jij) that satisfy these conditions are 
equal probability. Let us define probability density func- 
tion 



p(Jn, Jij, Jnn) 



(65) 



From symmetry conditions that define this function 
(p) with respect to transpositions lines and columns in 
matrix (Jij) we can conclude that this function (p) also 
doesn't transform with respect to such transpositions. 

Let us consider function p u /ij(u/ij) which is a discrete 
version of the function pj( a ,b)/a,b(J( a ,b)/a,b) : 



Pu/ij(u/ij)= I ... / p(Jll,...,J n k,--,Jij=U,...,J NN ) 



(66) 



Let us transpose lines and columns (Jij) by such way 
that element Jij will be replaced by element J n k , the 
function p(Ju, ...) will not be transform after it. So from 
previous equation we obtain 



Pu/ij{u/ij) 



4-oc 



Yl dJim = p u/nk (u/nk) 

(lm)=£(nk) 



(67) 



From this equation we can conclude that p u /ij( u /ij) 
doesn't depend on ij so pj/ a b(J/ a b) doesn't depend on 
ab and 



(68) 



And 



P.j/ab(J/ab) = pj(J) 

r+oo 

F[J(a,b)] = pj(J)JdJ = Const (69) 

Jo 



From 



1 /■! 



J(a, b)dadb = 1 



(70) 



'o jo 
we cad conclude that 



[ [ F[J(a,b)]dadb = 1 (71) 
Jo Jo 

So we can obtain that Const = 1 in Eq.(M 



B. Lemma 2 

Probability distribution functions a and (3 doesn't de- 
pendent on F\ and F2. 



Pa/F 1 (a/F 1 ) = p a (a) 
P0 /F2 (P/F 2 )=p (p) 



(72) 
(73) 



Proof: 

Let us make sampling of function a(Fi) by dividing of 
domain of tis function F\, [0, 1] on intervals of 1/N, N 3> 
1 Then restriction conditions for a^, k = 1, N: 



1 N 



Otk 



k=l 



< a k < 1 

% ggiGPi) 

dF 1 



dF x = 9 



(74) 



(75) 



All columns (ak) that satisfy by this conditions are 
equal probability. Let us to consider respective func- 
tion p(a±, ak, ai, czn)- From symmetry condi- 
tions that define this function with respect to transpo- 
sitions ak — > ai function p(a\, a&, a;, apf) also 
doesn't transform with respect to such transpositions. So 
we can write 

Pk(u) — i p(ai, a k = u, ai, a N ) da n — 

I p(ai, a k , ...,ai = u, ...,a N ) TT da n = 
Jo 



p(Jn, Jnk — u, Jij, Jnn) Pi(u) 



(76) 



From this equation we can conclude that function 
Pa/FA a l F\) doesn't depend on F\. 

Pa/ 'Fi ("/ Fi) = p a (a) (77) 

From (^9] ) we obtain 

F[T 2 (a, (3, 6)C - 2T(a, (3, 9)D] = 
1 ,1 

/ p a (a)pp((3)dad(3 
Jo 

[ I PH l/a .F 1 ( H i/a,F 1 )p H2/ ^ F2 (H 2 /l3,F 2 )dH 1 dH 2 
Jo Jo 

Jaf3 J(l-a)(l-/3). 



OO POO 



Jo 



pj(J)pj(J)[r'(a,(3,9)[- 



-2T(a,(3,9)'^-]dJdJ 



Pa(a)pp(f3)dad/3 



'0 Jo 

I? 2 (a, P,9)[ 



2, a n^ F \W , F[J\(l-a)(l-0), 



1 - 1 



-2T(a,j3,6) 



F[J]af3, 



(78) 



5 



Let us define 



v _ <*P | (l-a)(l-/3) 



(79) 



(80) 



By Lemma 1 , F[J) = F(J] = 1. Hence 
F[T 2 {a,(3,9)C - 2T(a, @,6)D] = f f [T 2 (a,(3,9)C 

Jo JO 

-2T(a,P,e)D}p a (a)p p ((3)dadp (81) 
It remain to find 



minr^frO) / / dFidF 2 
Jo Jo 

/ dad/3p a (a)pp(/3) 
'o Jo 

[T 2 {a,(3,9)C -2T(a,P, 9)D] 



Since 



Pa (a)p (l3)>O 



(82) 



(83) 



if the expression in square brackets is minimized at 
each point then the whole integral in ( |S2"| ) is minimized. 
Thus we may proceed as follows 



d_ 

or 



[r 2 (a, 0, 9)C - 2Y(a, j3, 9)D] = 2T(a, j3, 9)C - 2D = 



(84) 



Hence the optimum T(a, /?, 9) is given by 



r opt (a,{3,6) = = 



a (J 



7] 2& -L 



VIII. MEAN DISTANCE BEETWEN THE 
PROPOSED APPROXIMATION OF P(A/X 1 ,X 2 ) ,- 
r(a,/3,9) AND THE ACTUAL FUNCTION 

P(A/Xi,X 2 ) 

The mean distance from ( |57| ) is 

DIS = F[\\r(a,p,9)-P(A/x 1 ,x 2 )\\} = 
i /-l 

Pa(u)pp((3)dadfJ 



o jo 

2/ 



[r 2 (a,/3,6')C-2r(a,^,6')L»] 
+Const 

where Const in this equation is defined by 



(86) 



Const = 

/+oo p+oc 
\ px 1 ,x 2 {x ll x 2 )[P{A/xi,x 2 )] 2 dxidx2\ (87) 
-oo J — oo 

From this equation we can find boundaries of the 
Const. From < P(A/x\, x 2 ) < 1 we can conclude 

Const < 

f + oo r + oo 

F\ I I p Xl ,x 2 {xi,x 2 )P(A/x 1 ,x 2 )dx 1 dx 2 ] 



oo J — OO 



= F[6] = 9 
The second condition is 

/+oo p + ca 
j Px 1 ,x 2 (xi,x 2 ) 
-OO J —oo 

[P{A/x 1 ,x 2 )-9] 2 dx 1 dx 2 ] = 

r + oo r + oo 

F\ I I px u x 2 {xi,x 2 ) 



(88) 



oo J — OO 



[P{A/x u x 2 y + 9 2 - 2P(A/x 1 ,x 2 )9)dx 1 dx 2 ] = 

r + oc r + oo 

F[ / p Xu x 2 {xi,x 2 )[P(A/x 1 ,x 2 )] 2 dx 1 dx 2 



— oo J —oo 



So from this two equations we can conclude 



< Const < 9 



(89) 



(90) 



By next step we would like find function p a {a) (ppifl)) 
in equation for DIS. 

Restrictions for function ce(Fi), < Fx < 1 are next: 
(i) 



(85) (ii) 



/ a(Fi)dFi = 9 
Jo 



< a(Fi) < 1 



(91) 



(92) 



In discrete form (for N — > oo) we can rewrite a set 
{ax,a 2 , a N } 
(i) 



1 N 



(93) 



(h) 



0<Q!i<l,i = l,2,.„,JV (94) 
Let us define function U(a set ) by next way 



A" 



U{a set ) = J2 a * for0<ai<l ,i = 1,2, N 



U(a se t) = +oo otherwise 



(95) 



G 



U(a set ) = ^ U i( a i) 



U t (a t ) = oti forO < a l < 1 
Ui(a>i) = +00 otherwise 



(96) 



(97) 



Then function that satisfy equal probability distribu- 
tion with considering restrictions (i),(ii) is 



p asct (a S et) = -S(U(a set ) - N8) 

where 6 - delta-function of Dirac. 
Constant C define by 



(98) 



+ OO /> + OG 



00 J — oc 



Pa act ( a set)dai...da N = 1 (99) 



It can be prooved (see each course of " Statistical me- 
chanics" ; transform from microcanonical to canonical 
distribution) that for N 1— > 00 distribution (|9|) is equal 
to next distribution: 



where Z and K can be found from equations 

f + OC /» + OG 



(100) 



/+00 p+oc 
■■■ Pa ae t( a set)dax... da N = 1 (101) 
-OO J — CO 



-f" OG Z' + OO 



[/(a sef )y9 aset (a se t)dai...daAT = iV6> (102) 



Quest function p a (a) can be find by 

f +00 



p a (a) = 

rj = 

D 
where 



From Eqs.(101),(h0a) we can find 



I = f K \* 
Z [ l-e- K> 



= A(K) 
where A(K) is decreasing function 



(103) 



(104) 



(105) 



(106) 



A(K) = 1 forK = -00 
A(K) = forK = +oc 
A(Jf) = 1/2 forK = 



HK) = 1- 



1 



otherwise 



(107) 



If A is root of Eq (refpor6) we can write from 
Eqs.(p^,p^),(^05|),([l06|) for function p a (a): 



For K = 

Pa{ a ) = 1 /orO < a < 1 

Pa (a) = otherwise 

For K = +00 

p a (a) = 26(a) < a < 1 

Pa(«) = otherwise 

For K = — oo 

p a (a) = 2<5(a - 1) 0<a<l 

Pa (a) = otherwise 

For otherwiseK 



1 



Pa (a) = ^ e 



-iCo 



< a < 1 
Pa (a) = otherwise 



where 2 J Q 5(a — 1) = 2 5(a) = 1 and 

A' 



1 
D 



-K 



(108) 



(109) 



IX. THE CASE OF MORE THEN TWO 
VARIABLES A AND X 

Let A be a random variable, with values in set 
0, 1,...,L. Assume that the apriori probability P(A = 
i) is known and denote it by 9^, here i = 1,...,L. 
Let X\, Xk be K random variables, with values in 
some set, say ] — oo;+oo[. We are given the follow- 
ing informational = Xi,...,Xk — Xk (obtained though 
measurement). Furthermore, we have two system - "clas- 
sifiers" , which given x\ ,...,xk produce: 



P(A = i/X j =x j ) 



(110) 



We wish to estimate the probability P(A = i/X\ = 
x\, Xk — xk) in terms of and 0j. More specifically 
we wish to find a function r opt M( a ij i 0») which on the av- 
erage is the best approximation for P(A = M/xi, Xk)- 
By the same way that in case of two variables we can find 
that the T op t t M(ctij,0i) defined by equation 



^opt,u( a ij j Oi) 



Eti(nf=i^)/ft 



iK-l 



(111) 
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We have evidential restraints for a,j,6i 



< aij < 1 



i=i 



(112) 



< 6i < 1 

X> = i 



(113) 
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Figures Legends 



Fig.l: Function T(a, f3, 6) : [0, l] 3 ^ [0, 1] 
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FIG. 1. The System 

Functionr(a,/?,0) : [0, l] 3 ^ [0, 1] 
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